Shape-based Evaluation of Epidemic Forecasts

Infectious disease forecasting for ongoing epidemics has been traditionally performed, communicated, and evaluated as numerical targets - 1, 2, 3, and 4 week ahead cases, deaths, and hospitalizations. While there is great value in predicting these numerical targets to assess the burden of the disease, we argue that there is also value in communicating the future trend (description of the shape) of the epidemic - for instance, if the cases will remain flat o r a s urge i s expected. To ensure what is being communicated is useful we need to be able to evaluate how well the predicted shape matches with the ground truth shape. Instead of treating this as a classification problem ( one out of n shapes), we define a transformation of the numerical forecasts into a "shapelet"-space representation. In this representation, each dimension corresponds to the similarity of the shape with one of the shapes of interest (a shapelet). We prove that this representation satisfies the property that two shapes that one would consider similar are mapped close to each other, and vice versa. We demonstrate that our representation is able to reasonably capture the trends in COVID-19 cases and deaths time-series. With this representation, we define an evaluation measure and a measure of agreement among multiple models. We also define the shapelet-space ensemble of multiple models as the mean of their shapelet-space representations. We show that this ensemble is able to accurately predict the shape of the future trend for COVID-19 cases and trends. We also show that the agreement between models can provide a good indicator of the reliability of the forecast. © 2022 IEEE.

Uncertainty quantification for epidemiological forecasts of COVID-19 through combinations of model predictions.

Scientific advice to the UK government throughout the COVID-19 pandemic has been informed by ensembles of epidemiological models provided by members of the Scientific Pandemic Influenza group on Modelling. Among other applications, the model ensembles have been used to forecast daily incidence, deaths and hospitalizations. The models differ in approach (e.g. deterministic or agent-based) and in assumptions made about the disease and population. These differences capture genuine uncertainty in the understanding of disease dynamics and in the choice of simplifying assumptions underpinning the model. Although analyses of multi-model ensembles can be logistically challenging when time-frames are short, accounting for structural uncertainty can improve accuracy and reduce the risk of over-confidence in predictions. In this study, we compare the performance of various ensemble methods to combine short-term (14-day) COVID-19 forecasts within the context of the pandemic response. We address practical issues around the availability of model predictions and make some initial proposals to address the shortcomings of standard methods in this challenging situation.

Forecasting of COVID-19 Epidemic Process by Random Forest Method

The new coronavirus has changed the life of the planet and continues to spread around the world. Mathematical modeling allows the development of effective scientifically substantiated preventive and anti-epidemic measures. Machine learning methods have the highest accuracy when constructing the predicted incidence of infectious diseases. In this work, a model of a random forest was built to calculate the predicted incidence of COVID-19. To verify the model, data on the incidence of coronavirus in Ukraine, Great Britain, Germany and Japan were used. These countries were chosen because have different dynamics of the epidemic process and different control measures. © 2021 IEEE.

Towards Equitable Futures in Frontline Health: Design of Intelligent Systems for Supporting (Gendered) Care Work in Resource-Constrained Settings

Frontline health workers in many countries are responsible for filling gaps in essential primary health infrastructure, as witnessed during the COVID-19 pandemic. Their work increasingly involves the use of purportedly "intelligent"systems or data collection for such systems, to support diagnosis, disease forecasting, and information delivery. My research aims to inform the design of data-driven and automated systems in frontline health work, particularly for women workers in low-level and precarious roles in the Global South. Drawing from literature in the fields of human-computer interaction (HCI), gender and development studies, and health informatics, I will critically examine health workers' experiences and relationships with "intelligent"systems, and engage in the participatory design of technology that might better serve worker needs while strengthening the frontline health ecology overall. © 2022 Owner/Author.

The EpiBench Platform to Propel AI/ML-Based Epidemic Forecasting: A Prototype Demonstration Reaching Human Expert-Level Performance

During the COVID-19 pandemic, a significant effort has gone into developing ML-driven epidemic forecasting techniques. However, benchmarks do not exist to claim if a new AI/ML technique is better than the existing ones. The “covid-forecast-hub” is a collection of more than 30 teams, including us, that submit their forecasts weekly to the CDC. It is not possible to declare whether one method is better than the other using those forecasts because each team’s submission may correspond to different techniques over the period and involve human interventions as the teams are continuously changing/tuning their approach. Such forecasts may be considered “human-expert” forecasts and do not qualify as AI/ML approaches, although they can be used as an indicator of human expert performance. We are interested in supporting AI/ML research in epidemic forecasting which can lead to scalable forecasting without human intervention. Which modeling technique, learning strategy, and data pre-processing technique work well for epidemic forecasting is still an open problem. To help advance the state-of-the-art in AI/ML applied to epidemiology, a benchmark with a collection of performance points is needed and the current “state-of-the-art” techniques need to be identified. We propose EpiBench a platform consisting of community-driven benchmarks for AI/ML applied to epidemic forecasting to standardize the challenge with uniform evaluation protocol. In this paper, we introduce a prototype of EpiBench which is currently running and accepting submissions for the task of forecasting COVID-19 cases and deaths in the US states and We demonstrate that we can utilize the prototype to develop an ensemble relying on fully automated epidemic forecasts (no human intervention) that reaches human-expert level ensemble currently being used by the CDC. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Forecasting of COVID-19 Epidemic Process by Lasso Regression

The coronavirus epidemic has changed the life of the whole world. Containment of the further development of the pandemic requires the implementation of effective evidencebased control measures. For this, it is advisable to use mathematical modeling. The most accurate predictions are shown by machine learning methods. The article discusses a lasso regression model for predicting the dynamics of a new coronavirus in Ukraine, Great Britain, Germany and Japan. The model shows high accuracy. The disadvantage of this approach is the impossibility of identifying the factors influencing the dynamics of morbidity. © 2021 UkrMiCo 2021 - 2021 IEEE International Conference on Information and Telecommunication Technologies and Radio Electronics, Proceedings. All rights reserved.

A Heterogeneous Ensemble Forecasting Model for Disease Prediction.

The manuscript presents a bragging-based ensemble forecasting model for predicting the number of incidences of a disease based on past occurrences. The objectives of this research work are to enhance accuracy, reduce overfitting, and handle overdrift; the proposed model has shown promising results in terms of error metrics. The collated dataset of the diseases is collected from the official government site of Hong Kong from the year 2010 to 2019. The preprocessing is done using log transformation and z score transformation. The proposed ensemble model is applied, and its applicability to a specific disease dataset is presented. The proposed ensemble model is compared against the ensemble models, namely dynamic ensemble for time series, arbitrated dynamic ensemble, and random forest using different error metrics. The proposed model shows the reduced value of MAE (mean average error) by 27.18%, 3.07%, 11.58%, 13.46% for tuberculosis, dengue, food poisoning, and chickenpox, respectively. The comparison drawn between the proposed model and the existing models shows that the proposed ensemble model gives better accuracy in the case of all the four-disease datasets.

Spatial prediction of COVID-19 epidemic using ARIMA techniques in India.

The latest Coronavirus (COVID-19) has become an infectious disease that causes millions of people to infect. Effective short-term prediction models are designed to estimate the number of possible events. The data obtained from 30th January to 26 April, 2020 and from 27th April 2020 to 11th May 2020 as modelling and forecasting samples, respectively. Spatial distribution of disease risk analysis is carried out using weighted overlay analysis in GIS platform. The epidemiologic pattern in the prevalence and incidence of COVID-2019 is forecasted with the Autoregressive Integrated Moving Average (ARIMA). We assessed cumulative confirmation cases COVID-19 in Indian states with a high daily incidence in the task of time-series forecasting. Such efficiency metrics such as an index of increasing results, mean absolute error (MAE), and a root mean square error (RMSE) are the out-of-samples for the prediction precision of model. Results shows west and south of Indian district are highly vulnerable for COVID-2019. The accuracy of ARIMA models in forecasting future epidemic of COVID-2019 proved the effectiveness in epidemiological surveillance. For more in-depth studies, our analysis may serve as a guide for understanding risk attitudes and social media interactions across countries.